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(57) Abstract: The invention relates 
to a method in which the information 
contents of an image of a moving object 
is enhanced. The invention also relates 
to a system in which such a method can 
be carried out and to a computer program 
enabling a data processing unit to carry out 
such a method. The method is used notably 
in the field of medical imaging systems. 
According to the method first a first image 
of a moving object is acquired by means 
of a first imaging method, said image 
containing artifacts which are caused by the 
object motion. From two further images, 
acquired by means of a second imaging 
method and representing the object in a 
respective state of motion of the motion, 
there is formed a motion model which 
is implemented in a different manner in 
image processing or image forming steps, 
so that the information contents of either 
the first image or a combination image, 
formed from the first image and the two 
further images, is enhanced. 
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Method and system for enhancing the information contents of an image 



The invention relates to a method of enhancing the information contents of an 
image of a moving object. The invention also relates to a system in which a method of this 
kind is carried out as well as to a computer program enabling a data processing unit to carry 
out such a method. This method is used notably in the field of medical imaging systems. 
5 A method of this kind is applied wherever images of a moving object are to be 

formed, which images often contain imavoidable motion artifacts. As a result, the object is 
usually imaged in a blurred fashion so that it offers a viewer only inadequate information 
conceming the object. Motion artifacts often give rise to unusable images in particular in the 
case of slice images or volimie images of a moving object. 

1 0 The article by D. Mattes et al. "Nomigid multimodality image registration". 

Medical imaging 2001: Image Processing, Proceedings of SPIE vol. 4322 (2001), discloses a 
method in which an image of a moving object, acquired and reconstructed by means of the 
PET method and containing motion artifacts, is superposed on a further image of the moving 
object, acquired and reconstructed by means of the CT method, so as to form a combination 

1 5 image. The combination image is formed by special registration of the two individual images 
while utilizing pregnant, bilateral similarity information contained in both images. Because 
of the physical circumstances, the PET image contains pronounced motion artifacts which are 
not taken into account in the disclosed method and give rise to problems during the 
registration. 

20 It is an object of the invention to increase the information content of images 

containing motion artifacts. 

This object is achieved in accordance with the invention by means of a method 

of enhancing the information contents which can be derived from a first image, containing 

motion artifacts, of a moving object, which method includes the following steps: 
25 a. using two further images which represent the object in a respective state of 

motion with as few motion artifacts as possible, 

b. determining a motion model which characterizes states of motion assumed by 

the object while performing the motion between the two states of motion. 
The first image may have to be reconstructed from projections. 
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There is a first image of a moving object which contains motion artifacts 



which are due to the motion of the object. Motion artifacts are the cause, for example, that the 
object is imaged less sharply as the object has moved more during the acquisition time. Such 
motion artifacts may occur when the acquisition time of the imaging method used for the 
acquisition is long in comparison with the motion, so that the object moves during the 
acquisition. The term "motion" is to be interpreted in a very broad sense. The object may 
perform, for example, a very complex natural motion (human heart) or merely a linear, 
uniform motion (a sphere rolling at a constant speed). 



these images represents a respective state of motion of the object and is as free from motional 
artifacts as possible, the two states of motions originating from the motion performed by the 
object during the acquisition of the first image. Alternatively, the two states of motions 
represented may also originate from a motion which has been performed by the object at a 
different instant and is at least approximately the same as the motion performed by the object 
during the acquisition of the first image. In order to image the states of motion with as few 
artifacts as possible, the acquisition time is generally short in comparison with the duration of 
the motion. 



and the two further images may be the same or be different. When a common imaging 
method is used for all images, the different characteristics of the first image aad of the two 
fiurther images can be realized by way of different adjustments of acquisition parameters. 
When two different imaging methods are used, each of which reproduces other 
characteristics of the object, information of such diflFerent characteristics can be 
advantageously supplemented at a later stage. 



characterizes states of motion assumed by the object while performing the motion between 
the two states of motion. In particular in the case of a complex natural motion of the object, 
individual parts of the object move differently during the execution of the motion; for 
example, some parts of the object travel only a short distance whereas other parts follow a 
long and possibly curved path. Starting from of one of the two states of motion, the motion 
model describes the behavior for different parts of the object during the motion from one 
state of motion to the other. 



Furthermore, at least two jfurther images of the object are involved. Each of 



The imaging methods that can be used for the acquisition of the first image 



There is also determined a motion model of the motion of the object which 
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The infonnadon thiis obtained as regards the motion of the object can be 
incorporated in various ways in supplementary image processing or image forming steps, 
resulting in a higher quality of the resultant images. 

For example, such supplementary steps in accordance with the invention as 
disclosed in claim 1 are: 

forming an intermediate image of the object from the motion model and the 
two further images, the intermediate image representing the object at least approximately as 
if it had performed the motion, 

d. forming a combination image from the intermediate image and the first image. 

The intermediate unage formed represents the object as if it were to perform 
the motion. To this end, starting from the two known states of motion, the object motion is 
imitated by means of the motion model and the intermediate image is formed from this 
information, for example, by superposition. The intermediate image thus reconstructed 
represents the object with substantially the same motion artifacts as the first image. 

A viewer may wish to view the first image with as few falsifications as 
possible, despite the motion artifacts contained therein, because information may be lost in 
the course of known image processing methods. The superposition of the intermediate image 
and the first image so as to form a combination image then offers the advantage that on the 
one hand the first image is presented with as few falsifications as possible whereas on the 
other hand the information from the two fiirfher images is prepared in the intermediate image 
in such a manner that it can be combined directly with the information of the first image. 

The combination can be carried out in a variety of manners. For example, the 
two images can be additively superposed, so that the relevant information of the mdividual 
images is represented in an overall image. The variable division of the combination image 
constitutes a fiirther possibility, a part of the combination image then being represented by 
the intermediate image whereas the other part is represented by the first image. A viewer can 
then move the boimdary between the two sub-regions, so that an object detail is represented 
once by the intermediate image and once by the first image. Alternatively, the sub-regions 
can also be displayed so as to be separated by a window, the window being displaceable 
across the combination image by means of a control imit. A simple method of forming the 
combination image consists in arranging the intermediate image and the first image adjacent 
one another so that the two images are simultaneously presented to the viewer. 

A fiirther supplementary step in accordance with claim 2 consists of: 
c. focusing the first image by means of the motion model. 
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During the focusing the motion artifacts are reduced to such an extent that the 



object represented in the first image is presented sharply to a viewer. Generally speaking, the 
object appears to be unsharp to a viewer because several states of motion of the object motion 
are represented in superposed form in an image. For the focusing of an object represented 



with motional unsharpness, the object is represented in only one state of motion of the motion 
in that the superposition with the remaining states of motion is eliminated. This is possible 
when the motion leading to the unsharp representation is known. The motion model of the 
object motion determined in the first step contains exactly this information concerning the 
object motion, and a focusing algorithm can utilize information from the motion model for 
focusing. This results in an image which represents the object in one state of motion (that is 
sharply) and appears to the viewer as if it has been acquired by means of the first imaging 
method. This is advantageous in particular when the first imaging method is not suitable, for 
example, for physical reasons, to acquire a sharp image of the moving object. 

When the first image is to be reconstructed from projections, in conformity 
with claim 3 there are the following supplementary steps: 

c. forming an mtermediate image of the object from the motion model and the 
two fiarther images, the intermediate image representing the object at least substantially as if 
it had performed the motion, 

d. reconstructing the first image from the projections of the object and tiie 

intermediate image. 

For this method to be used it is necessary that, instead of a first unage, 
projections of the movmg object are available first and that the first image is reconstmcted 
from these projections, for example, by means of a data processing unit. Methods of this kind 
are known inter alia from the field of medical slice or volume unking, such as computed 
tomography, magnetic resonance tomography or positron emission tomography. 



case of positron emission tomography, the fact must be taken into account that a part of this 
radiation is absorbed by other parts of the object. Methods are known which take into account 
information concerning such so-called location-specific attenuation for the reconstruction in 
order to avoid artifacts in the reconstructed image. To this end, usually a transmission image 
of the object if formed, which transmission image represents the necessary location-specific 
attenuations. It is a drawback, however, inter alia that the motions of the object, leading to 
artifacts in the first hnage, are not or not adequately present in the transmission image so that 



In particular in the case of radiation-emitting object, as used for imaging in the 
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they cazmot be taken into account for the reconstruction. This leads to artifacts in the 

reconstructed image. 

From two points of view it is particularly advantageous to utilize the 

intermediate image during the reconstruction: on the one hand, the intermediate image 
5 contains the necessary information for location-specific attenuation and on the other hand the 

motion of the object, being present in the intermediate image, is taken into accoxmt. The 

images reconstructed with the aid of such an intermediate image have an enhanced quality in 

comparison with images reconstructed by means of conventional transmission images. 

Therefore, the intermediate image should generally contain information concerning the 
10 formation of the image by means of the first imaging method, in this case being information 

as regards the location-specific attenuation. 

It is to be noted that in particular the methods in conformity with the claims 1 

and 3 or the claims 2 and 3 can be combined and that they advantageously supplement one 

another. 

15 In particular in the case of objects whose parts move differently the 

representation by way of motion vector fields in conformity with claim 4 offers a simple and 
adequate location-specific representation of the necessary information for the remainder of 
the method. A motion vector field indicates how or along which path the corresponding part 
of an object moves while performing the motion between the two known states of motion. 

20 When it is known how ofl;en the relevant states of motion are assumed by the 

object while performing the motion, or how long the relative stay is of the object in the 
relevant state of motion, the generating of the intermediate image is particularly simple in 
conformity with claim 5. Such information can be based, for example, on a model of the 
miction or be determined by means of a sensor while the motion is performed. Such formation 

25 of the intermediate image offers the advantage that it is not necessary to describe the entire 
motion in the motion model, but only information concerning individual states of motion as 
well as their firequency. 

Because of the use of different imaging methods it may occur that the 
individual parts of the object in the intermediate image are differently localized relative to the 

30 first image. Relative differences between the object parts themselves as well as absolute 
differences relative to the image edges or ratio of dimensions of the object parts may then 
occur. In other words, the object is represented in the intermediate image with a co-ordinate 
system which differs firom that in the first image. The registration as disclosed in claim 6 
enables the relevant parts of the object to be transformed either firom the positions 
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represented in the intermediate image to the positions represented in the first image or vice 
versa. All parts of the object are then in the same image positions in the intermediate image 
as well as in the first image, thus enabling superposition of the two images. 



5 represented in the combination image to be represented in a selected state of motion of the 
motion. To this end, known focusing algorithms are used in combination with the motion 
model and/or one of the two further images. The state of motion then represented can 
correspond, in conformity with the focusing algorithm used, any state of motion of the 
motion, but notably the states of motion represented in the two further images. 

10 An advantageous further version of the method of claim 2 is disclosed in cMm 

8. A viewer is not only offered the sharper first image, but additionally a comparison with 
one of the two further images is made possible. 

The method in accordance with the invention can in principle be carried out by 
means of two imaging methods which are based on the same modality. As opposed thereto, 

15 the use of different modalities enables the representation of different characteristics of the 
moving object in the corresponding images. The method in accordance with the invention is 
advantageously used in conformity with claim 9 when the first imaging method cannot offer 
images of a moving object without motion artifacts. 

The object is also achieved by means of the system as disclosed in claim 10. 

20 An image processing system is to be understood to mean any system capable of receiving the 
images or the data formed by the method in accordance with the invention, of processing 
these images or data accordingly, and of either visualizing the result or applying it to other 
systems. The image processing system may be completely independent firom the devices 
acquiring the image data by means of the relevant imaging methods. On the other hand it is 

25 also feasible to constract the image processing system as a component of a larger system. The 
data processing \mit may optionally be constructed so as to be programmable. 

The object is also achieved by means of an examination system as disclosed in 
claim 1 1 . The devices for forming the images are well known from prior art so that they will 
not be elaborated herein. By way of example there may be mentioned: X-ray systems, 

30 magnetic resonance tomography apparatus and apparatus from the field of nuclear medicine. 
The two imaging methods may be realized by means of a common modality of by means of 
different modalities. The difference will be illustrated on the basis of the following example: 
a common modality is assimied to be a conventional X-ray fluoroscopy where the first 
imaging method produces images containing motion artifacts during a long acquisition time 



An additional step in accordance with the invention enables the object 
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and with a low dose, and the second imaging method produces images during a short 
acquisition time and with a higher dose. Depending on the construction of the examination 
apparatus, the two imaging methods can be carried out by means of the same device by 
variation of the relevant parameters. When different modalities are used, one device may be, 
5 for example, a computed tomography apparatus and the other device a PET system. 

If the data processing unit of a system as described above is constructed so as 
to be programmable, a computer program as disclosed in claim 12 enables the data 
processing \mit to carry out a method in accordance with the invention. Depending on the 
configuration, the computer program can be made available to the data processing unit by 
10 means of a computer program product such as an external, portable storage medium. 



The following examples and embodiments are illustrated with reference to the 
Figs. 1 to 6. Therein . 
1 5 Fig. 1 shows the block diagram of a version of the method in conformity with 



claim 1, 
claim 2, 
20 claim 3, 



Fig. 2 shows the block-diagram of a version of the method in conformity with 
Fig. 3 shows the block diagram of a version of flie method in conformity with 



Fig. 4 shows, by way of example, the states of motion of human respiration. 
Fig. 5 shows the state of motion function of a uniform motion, and 
Fig. 6 shows a PET-CT combination system. 

The Figs. 1 to 3 show the steps of a number of versions of the method in 
25 accordance with the invention. The rectangular boxes represent results, data, images etc. 
Steps of the methods are stated within an ellipse. 



Fig. 1 is a diagrammatic representation of the steps and results of a first 
30 version of the method. It is an object of the method to acquire images of a moving object by 
means of two different imaging methods, in this case being methods based on PET and CT. 
The different imaging methods enable different information to be acquired as regards the 
moving object. Such different information is to be presented in conmion to a user. 
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Projections PI which have been made, for example, of the thorax of a patient 
at the area of the diaphragm by means of the PET (Positron Emission Tomography) method 
are available (the method itself is not shown in Fig. 1). The PET method is known from the 
field of nuclear medicine and is intended for the formation of slice images or volume images. 
A metabolic preparation, marked with given, unstable nuclides, is then injected into a patient, 
said preparation being taken up in a tissue-specific or function-specific manner. The radio 
nuclides used decay, giving rise to two y quanta in different successive processes in the 
vicinity of the location of decay, which quanta take off in exactly opposite dkections, leave 
the patient and can be detected by smtable sensors which are arranged in a detector in the 
form of a ring around the patient. On their travel firom their location of origin to Iheir point of 
emergence firom the patient the y quanta traverse fardaei tissue of the patient which can 
absorb the y quanta more or less as a function of the type of tissue. Generally speaking, tiie y 
quanta are attenuated in a tissue-specific manner. The whole of detected y quanta forms a set 
of projections PI of tiie object M^ierefirom a slice image or volume image can be 
reconstructed in known manner during a subsequent reconstruction. The PET method yields 
functional images of the object. 

During the acquisition of tiie projections PI, which may last from a few 
minutes to one hour, the patient performs a respiratory motion during which the diaphragm 
moves m conformity with the respiration. This respiratory motion causes motion artefacts in 
the reconstructed PET image 10, said artifacts becoming manifest as an unsharp and blurred 
image of the object. At least two causes of such motion artifacts are known: 

1 ) Because of the respiratory motion, a given location of a tissue occupies 
different positions relative to the detector, so that y quanta arising at this location are acquired 
by different sensors of the detector. 

2) Y quanta which successively arise in substantially the same location are 
attenuated to a different extent, because tiie relative position of the surrounding tissue, which 
is responsible for the attenuation, changes relative to the location of origin of the X-ray 
quanta due to the respiratory motion. 

Also available are the imj^es 12 and D vMch have been acquired by means of 
tiie CT-based metiiod which is not shown in Fig. 1, The CT (Computed Tomography) metiiod 
is known, for example, fi-om tiie medical field and scarves to form slice images and volume 
images of objects or patients, such images containing anatomical information. A CT-based 
system will be described in detail hereinafter. As opposed to PET images, CT images contain 
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substantially fewer or no motion artifacts, because the image acquisition can take place 
substantially faster in relation to the motion. Known systems, combining PET images with 
functional information on the object and CT images with anatomical information on the 
object, ignore the motion artifacts of the PET images upon combination of the information, 
so that the combined information includes further artifacts or errors. These artifacts or errors 
are significantiy reduced by means of the invention. In the further course of this description it 
will be demonstrated how artifacts which arise in the PET image due to the above cause 1) 
are taken into account in the combined information. 



example, a combination system which consists of a computed tomography apparatus and a 
PET system. The computed tomography apparatus and the PET system are both configured in 
principle as independent systems which, however, are geometrically coupled relative to a 
common axis of reference. For the image acquisition the systems are usually used 
consecutively; for example, first CT images, representing two striking states of motion, are 
acquired and subsequentiy the acquisition of the PET data takes place. 



rotation around an axis of rotation 14 which extends parallel to the z direction. To this end, 
the gantry 1 is driven at a preferably constant but adjustable angular speed by a motor 2. A 
radiation source S, for example, an X-ray tube, is attached to the gantry 1 . The radiation 
source is provided with a collimator arrangement 3 which forms a conical radiation , beam 4 
from the radiation produced by the radiation source S. The radiation beam 4 penetrates a . 
moving object (not shown) which is present in a cylindrical examination zone 13. After 
having traversed the examination zone 13, the X-ray beam 4 is incident on a two-dimensional 
detector xmit 16 attached to the gantry 1 . 



defined as the angle enclosed by a ray of the beam 4 which is situated at the edge in the x-y 
plane relative to a plane defmed by the radiation source S and the axis of rotation 14) then 
determines the diameter of the examination zone 13 within which the object to be examined 
must be present during the acquisition of the measuring values. In order to generate volume 
images of the object, the object which is arranged, for example, on a table in the examination 
zone 13, can be displaced parallel to the direction of the axis of rotation 14 or the z axis by 
means of a motor 5. The measuring data acquired by the detector unit 16 is applied to a 
reconstruction imit 10 which reconstructs therefrom the absorption distribution in the part of 
the examination zone 13 which is covered by the radiation beam 4. The two motors 2 and 5, 



In order to illustrate the acquisition of such images. Fig. 6 shows, by way of 



The computed tomography apparatus includes a gantry 1 which is capable of 



The angle of aperture amax of the radiation beam 4 (the angle of aperture is 
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the reconstruction unit 10, the radiation source S and the transfer of the measuring data from 
title detector unit 16 to the reconstruction unit 10 are controlled by means of a suitable control 
unit?. 

The motors 2 and 5 can be controlled in such a manner that the ratio of tihie 
speed of propagation of the examination zone 13 to the angular velocity of the gantry 1 is 
constant, so that the radiation source S and the examination zone 13 move along a helical 
path, that is, the so-called trajectory, relative to one another. In this case it is urelevant 
whether the scanning unit, consisting of the radiation soxirce S and the detector unit 16, or the 
examination zone 13 performs the rotary motion or the propagation motion, because only the 
relative motion is of importance. The object is not displaced for the formation of slice 
images. 

A motion signal is derived by means of an acquisition unit 12 and a motion 
sensor 15 which is arranged on the object in order to detect the object motion. This signal can 
be applied, if desired, to the reconstmction imit 10 in order to facilitate the selection of the 
measuring data that is suitable for the reconstruction. Furthermore, the motion signal is used 
(as will be described in detail hereinafter) in accordance with the invention for the 
determination of the motion model. 

Fxirthermore, a PET acquisition unit 20 is arranged so as to be concentric with 
relative to the axis of rotation 14; tihis unit is arranged as a ring around the object which is 
present in the examination zone 13. The acquisition unit 20 comprises individual sensors 21 
which detect the y quanta emitted by the object. In order to form slice images, it suffices to 
configure the PET acquisition unit so as to be quasi two-dimensional by arranging sensors 21 
adjacent one another in a ring-like configuration. In order to generate volume images, the 
PET acquisition unit comprises a plurality of such rings of sensor which are arranged so as to 
be parallel to one another and around the axis of rotation 13. The signals detected by the 
sensors 21 are applied to an evaluation unit 22 which forms one or more PET images 
therefrom by means of known algorithms. 

In addition to the above-mentioned functions, the control unit 7 is also 
arranged to displace the object between the acquisition positions of the CT system as well as 
those of the PET system. After successful acquisition of the CT data, for example, the object 
on the table is displaced, by means of the control unit 7 and the motor 5, to the acquisition 
position of the PET acquisition unit 20 in which the relevant preparation is injected into the 
object and the PET data is acquired. 
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The data processing unit 23 is arranged to combine, using the method in 
accordance with the invention as well as the motion signal, the information contents of the 
CT images and the PET images, to reduce motion artifacts, if any, and to visualize the results 
accordingly by means of a display apparatus 1 1 . If the data processing unit is constructed so 

5 as to be programmable, it is enabled by a computer program to carry out the method in 

accordance with the invention. The computer program may be stored in an internal memory 
such as, for example, a ROM or an EPROM, or in a computer program product such as a disc 
or CD-ROM provided with the computer program. 

The computed tomography apparatus shown as well as the PET acquisition 

10 xmit may be configured in such a manner that slice as well as volume images can be acquired. 

The images 12 and 13 of Fig. 1 represent two different states of motion of the 
motion performed by the object dvuring the acquisition of the projections PI. Because it is 
inherent of the method of CT-based acquisition that significantly less time is reqiiired than 
for acquisition on the basis of PET, instantaneous images can be formed of the respiratory 

1 5 motion of the patient, notably the image 12 of the inhaled state and the image 13 of the 

exhaled state. The inhaled as well as the exhaled state of motion of the patient represents a 
particularly characteristic state of the respiratory motion. Alternatively, a sequence of CT 
images IS of the respiratory motion can be acquired during which the object performs the 
same motion as during the acquisition of the projections PL Subsequentiy, using known 

20 methods, the two images 12 and 13 are extracted from the sequence IS. This offers the 

advantage that the motion more accxirately corresponds to the motion performed by the object 
during the acquisition of the projections PI . When the images 12 and 13 are acquired directiy, 
often unnatural or cramped states of motion occiir in the patient; such states are not the same 
as those occurring during tiie natural respiratory motion. 

25 From these state of motion images 12 and 13 an object motion model M2 is 

formed in the step CI in combination with a state of motion function Fl . A state of motion 
function Fl of this kind is shown in Fig. 4, by way of example, as a frequency function f(r) 
for the human respiratory motion. The y axis of the frequency function f(r) describes how 
often the relevant states of motion are assumed by the object while it performs the motion. 

30 The x axis represents successive states of motion r assumed while the motion is performed. In 
other words, the frequency function describes how long on average the object stays in a given 
state of motion in relation to the other states of motion while the object performs the motion. 
If the representation is normalized, the integral over this curve produces exactly 1 . It is a 
particularly significant aspect that during the respiratory motion the organs or tissue involved 
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in the respiratory motion remain particularly frequently or long in the exhaled state 
(exhalation). If merely the frequency is of interest for the ftirther steps of the method, the 
order of the states of motion assumed while the object perforais the motion can be ignored 
and the states of motion which are plotted on the x axis need not necessarily succeed one 
another during the execution of the motion. 



only provides information as regards the frequency of the assumed states of motion, but also 
describes the overall motion process (states of motion as well as their frequency and 
completion in time). Because such a state of motion function is redundant for the use in the 
next steps of this version, however, it suffices to utilize a frequency ftuiction as described 
above. 



from a fiinction model Ml without taking into account the actual motion. This is possible . 
notably when in most objects a motion process is very similar or the same. For example, 
when in the case of a hviman respiratory motion the model Ml is formed by the frequency 
fimction f(r) of Fig. 1, it can be considered directly as the state of motion function Fl and be 
used for determining the motion model M2. Alternatively, the actual motion process can be 
determined by means of an appropriate sensor S 1 . In the case of the respiratory motion, for 
example, the respiratory motion is determined during the acquisition of the PET projections 
PI by means of a respiratory motion sensor provided on the patient and ttierefrom the 
frequency function f(r) or, more generally, a state of motion function Fl is derived for the 
construction of the motion model M2. 



the state of motion function Fl. Each of the images 12 and 13 represents a respective known 
state of motion of the object, said states of motion also being present in the state of motion 
function Fl. Assuming that all components of the object move locally linearly during the 
motion process, that is, along straight paths of different direction and length, between the 
states of motion shown in the images 12 and 13, for each pixel represented in one of the 
images 12 or D or for each object component the execution of the motion can be determined 
by means of the state of motion function Fl . This location-specific motion process can be 
retained in a motion vector field and constitutes, in conjunction with the state of motion 
function Fl, the motion model M2. A motion vector thus indicates the direction in which and 
the speed at which or the distance over which a pixel or an object component moves dxiring 
the execution of the motion between each time two states of motion. 



Altematively it is also feasible to utilize a motion state function Fl which not 



For known motion processes the state of motion function Fl can be derived 



In the step CI a motion model M2 is derived from the images 12 and 13 and 
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Mathematically speaking, a vector field of this kind can be described as 
jcj = 5C3 + rh(x^) . A point X3 moves towards the point during the execution of the motion; 
the motion is characterized by the motion vector field rh(x^) and each state of motion can be 
approximated by ic(r) = -h rm(x^) . The parameter r € [0,l] represents for r = 0 the state of 

5 motion represented in the image 13, where x(r = 0) = ^3 and for r = 1 the state of motion 
x(r = 1) = shown in the image 12. For the human respiratory motion, for example, 
describes the exhaled state and X2 describes the inhaled state. Generally speaking, such a 
mathematical description can also be applied to a variety of other motions. The motion need 
not be inherently periodic and be repeated. Fig. 5 shows the frequency function of a motion 

10 of an object in which all states of motion are equally frequently assumed during the execution 
of the motion. This is the case, for example, for the motion where all parts of the object move 
in the same direction at a constant speed (for example, a bicycle rider). 

The next step C2 aims to form from the images 12 and 13 an image 14 which 
exhibits substantially the same motion artifacts as the image 10. This is achieved in that, 

1 5 starting from one of the images 12 or 13, first artificial images of the remaining states of 
motion are formed by means of the motion model M2, said artificial images then being 
superposed so as to form the image 14: 

im = ]nr)I3(x, ■^rfh{x,)yir . 
0 

The image 14 is formed from the integral over all states of motion r of the product of the 
20 image 13 and the motion vector field {x^ + fh{x^)) weighted by the frequency function f(r), 
the motion vector field (3C3 + in{x^y) and the frequency function f(r) constituting the motion 
model M2. As a result, the image 14 represents a superposition of dX\ states of motion 
weighted by the relevant duration of stay. 

Analogously, a theoretical consideration of the formation of the PET image 10 
25 leads to the result that the PET image 10 also constitutes a superposition of all states of 

motion weighted by the relevant duration of stay; this means that it is also formed from the 
integral over all states of motion r of the product of an image lOa (not shown in this case) of 
an initial state of motion, the frequency function f (r) and the motion model (x^ + wi(x^)) : 

/0(x)= j/(r)/0,(x„ +rm(xj)rfr. 



pTt/i 
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In the ideal case the image 10a represents the same state of motion as the 
image 13 and the vectors and as well as the relevant frequency function and the motion 

model correspond. This consideration of the formation of the image 10 is used in the focusing 
step described hereinafter, because the appearance of the motion artifacts in the image 10 is 
5 thus substantiated. 

As an alternative for a combination apparatus as shown in Fig, 6, the images 
10 as well as the images 12 and 13 can be acquired by means of two independent systems. The 
object is then positioned in the first apparatus, for example, the PET system, and the PET 
image is acquired. Subsequently, the object is positioned in the second system and the CT 

10 images are acquired. Because of the local separation of the two systems, the object 

components in the PET image 10 usually are localized differently from those in the CT 
images 12, 13 and 14, so that only an inadequate direct superposition or direct comparison of 
the images 14 and 10 can be performed. Therefore, in a step R2 a so-called registration 
operation is performed. The step R2 is optional and is not necessary when the co-ordinate 

15 system of the images 10 suitably accurately corresponds to that of the images 12 and 13 (as is 
the case in the system shown in Fig. 6). 10 can then be equalized with 15 and the method can 
be continued as described hereinafter. 

Registration is a generally known method of equalizing co-ordinate systems 
between two or more images witii corresponding structures; a physical correspondence of 

20 two identical objects of different contents may then also occur. This is the case, for example, 
for a fimctional PET image and an anatomical CT image of the same object. After successftil 
registration, the object parts represented in one image can be associated with the 
corresponding object parts in the other image by the co-ordinate transformation determined. 
As a result, for example, the object parts shown in one image can be shifted in such a manner 

25 that they occupy the corresponding positions of the identical object parts represented in the 
other image. 

The article by D. Mattes et al. **Nonrigid multimodality image registration" , 
Medical Imaging 2001: Image Processing, Proceedings of SPIE Vol. 4322 (2001) describes 
how such a registration of a CT image and a PET image can be carried out. Because the 
30 registration is generally known, it will not be fijrther elaborated herein. It is to be noted, 
however, that in order to facilitate the registration in known PET systems, a transmission 
image is acquired in addition to the actual PET image as is also mentioned in the cited article. 
Prior to the injection of the metabolic preparation marked with nuclides, a radioactive source 
is displaced along a trajectory around the patient for this purpose, said source emitting rays in 
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the direction of the patient which penetrate the patient and are detected by sensors of the PET 
detector which are situated opposite the radiation source. The transmission image 
subsequently reconstructed is in principle similar to a CT image and, because of the 
anatomical information contained therein, is better suitable for registration with the CT 

5 image, notably in the case of a poor image quality of the actual PET image. When the co- 
ordinate transformation between the transmission image and the CT image has been 
determined, the co-ordinates of the actual PET image and the CT image can be made to 
correspond, because the co-ordinate systems of the transmission image and the PET image 
are substantially identical. This method is not shown in Fig. 1, but can nevertheless be used 

10 herein. 

After successful registration R2, the pixels or the object components of the 
image 10 can be transformed in the co-ordinate system of the image 14. This produces the 
PET image 15. A possible alternative, that is, transforming the image 14 into the co-ordinate 
system of the image 10, will not be elaborated herein. A comparison or a combination of the 

15 images 14 and 15 can now be performed, because both images exhibit substantially the same 
motion artifacts of the object motion and both images represent the object in relation to the 
same co-ordinate system, A viewer, for example, a physician, may desire the presentation of 
the image 10 without falsifications (a co-ordinate transformation in this sense does not 
represent a falsification) for given applications so as to enable comparison or combination of 

20 information thereft-om with information from other images. Prior art merely offers the _ 
possibility of comparing or combining only one of the two images 12 or 13 directly with the 
image 10. As opposed to the image 10, the images 12 or 13 do not exhibit motion artifacts, so 
that on the one hand the searching of a suitable co-ordinate transformation is more difficult 
and on the other hand the combination or comparison itself is possible only with difficulty. 

25 Such comparisons or combinations as shown as the step R3 in Fig. 1 are 

known in principle and can be realized, for example, by superposition, by adjacently arranged 
images, by partial superposition with manually selectable boundaries or by partial imaging of 
one image in the other image with manually selectable boundaries. The result of the step R3 
constitutes the image 16. 

30 In dependence on the relevant application, it may be useful for a user of the 

system to limit the image 16 to given information or to highlight given information in 
particular. To this end, an optional step Bl can be performed for focusing the image 16 which 
contains artifacts and in which only one state of motion of the motion is represented. A 
known algorithm for focusing which is suitable for linear motions is described in the article 
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by A.K. Katsaggelos „Iterative image restoration algorithms", OPTICAL ENGINEERING, 
July 1989, Vol. 28, No. 7, p. 735 ff. Notably the equation (1) shows how motional 
unsharpness can in general be expressed by modeling the motion to a linear motion and be 
compensated in the further course of the method. 

5 Fig. 2 shows a further version of the method in accordance with the invention 

in a combination system. As in Fig. 1, again a motion model M2 is determined and also a 
PET image 10 is formed. The registration R2 shown in Fig. 1 is dispensed with, becaxise the 
correspondence of the co-ordinate systems of the images 10, 12 and 13 is adequate. 

In a step B2, analogous to the step Bl of Fig. 1, focusing of the image 10 is 

10 carried out. As opposed to the focusing in Fig. 1, the focusing B2 of Fig. 2 is carried out in 
such a manner that the sharper PET image 18 represents the same state of motion as an 
acquired CT image, in this case the image 12. As a result, the image 18 can be compared or 
combined directly with an original CT image, in this case being the image 12. Alternatively, 
this process can also be carried out by means of the image 13 or an image from the image 

15 sequence IS. In the comparison step R4 the same methods are used as described with 
reference to Fig. 1, step R3. The image 19 is the resultant image. 

An alternative for the method of Fig. 2 (not shown) is to apply the focusing Bl 
to the image 15 instead of to the combination image 16 in Fig. 1 when the co-ordinate systems 
of the image 10 and the images 12 and 13 are dijfferent. As a result of the previously executed 

20 registration R2 it is first achieved that the co-ordinate system of the image 15 corresponds to 
that of the image 14 and hence to that of the motion model M2. This enables focusing Bl of 
the image 15 by means of the motion model M2. The resultant image is a PET image which 
represents a state of motion of the motion and contains practically no motion artifacts. 

Fig. 3 shows a further possibility for enhancing the information contents of an 

25 image by means of the motion model. In known PET systems a transmission image is 

acquired by means of the above method, that is, in addition to the actual PET image. This 
transmission image represents the location-specific attenuation of the radioactive radiation. 
This attenuation information is used for the reconstruction Rl of the projections PI in order 
to correct the absorption of y quanta by surroimding tissue (attenuation correction). In the 

30 article by I.T. Hsiao et al. „Noise Propagation from Attenuation Correction into PET 

Reconstructions" (pubUshed in „Nuclear Science & Medical Imaging including Nuclear 
Power Systems, 2000 Symposium", IEEE, ISBN 0-7803-6503-8) various methods are 
disclosed, for example, in Table 1, for the correction of the location-specific attenuation. 
Equation (4) reveals a possibiUty for multiplying the reconstruction, expressed as a linear 
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reconstruction operator, by the location-specific attenuation infonnation while taking into 
account distribution efifects. 

These methods, however, have the drawback that the transmission image, as 
opposed to the actual PET image, often contains only few or even no motion artifacts, 
5 becaiise the acquisition time is short in comparison with the duration of the motion. As a 
result, tiie attenuation information is not appropriately used and the motion artifacts in the 
PET unage, due to the above-mentioned cause 2, are not taken into account. 

Therefore, in accordance with the invention the reconstruction R5 of the PET 
image 110 utilizes a transmission image 14 which is formed, as in Fig. 1 , from the CT images 
10 12 and 13 as well as the motion model M2. The image 14 contains substantially the same 

motion artifacts as a PET image which is reconstructed without any attenuation iiiformation. 
As a result, the motion artifacts in the PET image are taken into account and the 
reconstruction R5 using the attenuation information from the image 14 produces a 
qualitatively enhanced image 110 in comparison with a reconstruction utilizing a 
15 conventional transmission image. 

Generally speaking, the methods of the figs. 1, 2 and 3 can also be 
simultaneously used in a system so as to enable presentation of the various results of the 
methods to a viewer. The method of Fig. 3 can be combined at will with the methods of Fig. 
1 and Fig. 2, that is, for as long as the system used is a combination system, because the co- 
20 ordmate systems of the PET system correspond substantially to those of the CT system .only 
in the case of a combination system. 

It is to be noted again that the methods shown in Fig. 1, Fig. 2 and Fig. 3 are 
not limited to CT and PET. For example, it is possible to acquire the image 10 by means of 
magnetic resonance tomography instead of a PET-based method or, alternatively, to acquire 
25 the images 12 and 13 by means of ultrasound or fast magnetic resonance tomography instead 
of by means of a CT-based method. Because tiiie method in accordance with the invention in 
general offers a possibility for comparing images containing different information on a 
moving object, while taking into account motion artifacts present in an image, applications 
outside the medical field are also feasible. For example, an image of a traveling car which 
30 contains motion artifacts can be acquired by means of a thermal-sensitive camera and further 
images can be acquired by means of customary photographic methods. The image of the 
thermal-sensitive camera shows location-specific functional processes, whereas the 
photographic images represent structural features of the object. The superposition of such 
different types of information so as to form a combination image while taking into accovmt 
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the motion ajiifacts advantageously enables the presentation of all information 
simultaneously to a viewer. 



• 
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CLAIMS: 



1 . A method of enhancing the information contents which can be derived from a 
first image, containing motion artifacts, of a moving object, which method includes the 
following steps: 

a. using two further images which represent the object in a respective state of 
5 motion with as few motion artifacts as possible, 

b. determining a motion model which characterizes states of motion assumed by 
the object while performing the motion between the two states of motion, 

c. forming an intermediate image of the object from the motion model and the 
two further images, the intermediate image representing the object at least approximately as 

10 if it had performed the motion, 

d. forming a combination image from the intermediate image and the first image. 

2. A method of enhancing the information contents which can be derived from a 
first image, containing motion artifacts, of a moving object, which method includes the 

15 following steps: 

a. using two fiirther images which represent the object in a respective state of 
motion with as few motion artifacts as possible, 

b. determining a motion model which characterizes states of motion assumed by 
the object while performing the motion between the two states of motion, 

20 c. focusing the first image by means of the motion model. 

3. A method of enhancing the information contents which can be derived from a 
first image, to be reconstructed from projections and containing motion artifacts, of a moving 
object, which method includes the following steps: 

25 a. using two further images which represent the object in a respective state of 

motion with as few motion artifacts as possible, 

b. determining a motion model which characterizes states of motion assumed by 

the object while performing the motion between the two states of motion. 
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c. forming an intermediate image of the object from the motion model and the 
two further images, the intermediate image representing the object at least substantially as if 
it had performed the motion, 

d. reconstructing the first image from the projections of the object and the 
intermediate image. 

4. A method as claimed in one of the claims 1 to 3, characterized in that a 

respective motion vector field is determined for parts of the object in order to determine the 
motion model. 



5. A method as claimed in one of the claims 1 to 3, characterized in that in order 
to form the intermediate image, first images of other states of motion of the object are formed 
by means of the two further images and the motion model, the images thus formed being 
weighted and subsequently superposed together with the two further images and in 

1 5 conformity with the frequency at which the respective state of motion represented in the 
images occurs while the motion is performed. 

6. A method as claimed in claim 1, characterized in that the intermediate image 
and the first image are registered, notably elastically registered, prior to the formation of the 

20 combination image. 

7. A method as claimed in claim 1, characterized in that the combination image 
is focused in a fiirther step. 

25 8. A method as claimed in claim 2, characterized in that a combination image is 

formed from the focused first image and one of the two fiirther images, possibly by means of 
registration. 



30 



9. A method as claimed in one of the claims 1 to 3, characterized in that the 

image is a PET image or a SPECT image and/or the two further images are CT images or, in 
the case of a method as claimed in one of the claims 1 or 2, MR images. 
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1 0. An image processing system which includes a data processing imit for 
carrying out a method as claimed in one of the claims 1 to 3 or a combination of these 
methods. 

1 1 . An examination apparatus, notably a medical examination apparatus, which 
includes 

a device for forming images and/or projections by means of a first imaging 



method, 
method, 



a device for forming images and/or projections by means of a second imaging 



an image processing system which includes a data processing unit for carrying 
out a method as claimed in one of the claims 1 to 3 or a combination of these methods. 



12. A computer program or a computer program product which is arranged to co 

operate with a data processing unit in such a maimer that the data processing unit can carry 
out a method as claimed in one of the claims 1 to 3 or a combination of these methods. 
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Remark on Protest | | The additional search fees were accompanied by the applicant's protest. 

I I No protest accompanied the payment of additional search fees. 



Form PCT/ISA/210 (continuation of first sheet (1 )) (July 1998) 



International Application No. PCT/ IB G3/ 03011 



FURTHER INFORMATION CONTINUED FROM PCT/ISA/ 210 



This International Searching Authority found multiple (groups of) 
inventions in this international application, as follows: 

1. claims: elms 1, 3, 5-7, elms 4. 9 when depending on elm 1 or 3 
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